This description relates to generation for sub-graph interfaces for dynamically linked sub-graphs.
Many software applications exist for processing data. Some of these software applications are specified as dataflow graphs. Dataflow graphs typically include a number of data processing components, which are interconnected by links, sometimes referred to as “flows.”
In operation, data (e.g., a dataset) is received by a dataflow graph from a database or from some other data storage system. The received data advances through the dataflow graph by propagating through the flows and into the components according to dependencies defined by the interconnection of the components and flows. Each component processes data that it receives according to a predetermined function associated with the component before providing the processed data as output data via one or more flows. At the output of the dataflow graph the processed data is, for example, stored in another data storage system, provided to another downstream system, or presented to a user.
A developer of a dataflow graph generally specifies the graph by dragging blocks representing components onto a graphical working area (or “canvas”) provided by a graphical development environment (GDE) and interconnecting the components with links representing data flows such that the dataflow graph implements a desired functionality. Once the developer is satisfied with their implementation of the dataflow graph they can save the dataflow graph to storage for later use. In general, if the developer needs to alter the their implementation of the dataflow graph at a later time, they cause the GDE to read the saved dataflow graph from storage, make changes to the dataflow graph, and then re-save the altered dataflow graph to storage.
In some examples, the components of a dataflow graph are themselves implemented using dataflow graphs which are referred to as “sub-graphs.” To alter a sub-graph of a component that is used in a given dataflow graph, the dataflow graph is read from disk, the sub-graph component is opened such that its sub-graph can be edited, changes to the sub-graph are made, and the dataflow graph itself is re-saved to storage, thereby embedding the changes to the sub-graph in the saved dataflow graph.